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Abstract We present an exaetly solvable random-subcube model inspired 
by the structure of hard constraint satisfaction and optimization problems. 
Our model reproduces the structure of the solution space of the random k- 
satisfiability and fc-coloring problems, and undergoes the same phase tran- 
. sitions as these problems. The comparison becomes quantitative in the 

large-fc limit. Distance properties, as well the ^-satisfiability threshold, are 
studied. The model is also generalized to define a continuous energy land- 
ed ' scape useful for studying several aspects of glassy dynamics. 
> 

' Keywords Constraint satisfaction problems • Clustering of solutions 

, Exactly solvable models 

00 

rn _ 

1 Introduction 

. Combinatorial optimization and Constraint Satisfaction Problems (CSPs) 

' arise in a wide array of scientific branches, including statistical physics, 

information theory, inference and machine learning. These problems, which 
T '"j ' involve a large number of variables interacting through a large number of 

K> I constraints or cost terms, are in general very hard to solve, and in most 
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cases no algorithm seems to be able to find a solution within a reasonable 
time, as formalized by the P ^ NP conjecture [1]. 

In order to circumvent this intrinsic difficulty and possibly to identify 
(or avoid) "hard" instances, random ensembles of optimization problems 
were introduced and used as test beds for theories and algorithms. This 
line of research has considerably benefited from the methods and concepts 
of statistical mechanics 2,3,4,5 . In particular, a spectacular breakthrough 
was made by the development of the survey propagation algorithm [5][6] 
which is able to solve large random instances of CSPs in the so-called 
"hard-SAT" region. The key to the success of the physics approach lies in 
the understanding of the rugged energy landscape (reminiscent of glassy 
phases) exhibited by these problems, which survey propagation exploits 
and integrates into a sophisticated message-passing [71 scheme. 

The structure and organisation of solutions has been analyzed in detail 
for several CSPs, including the satisfiability problem (fc-SAT) |8j|, the col- 
orability of random graphs (A;-COL) [9ll8lll0j. and systems of linear Boolean 
equations (fc-XORS AT) pT[[T2l [T3j . It was shown that as the density of con- 
straints is increased the space of solutions undergoes several phase transi- 
tions. At low density of constraints the solution space is concentrated in 
one big ergodic component, called "cluster" or "state". For higher densi- 
ties the systems undergoes a clustering transition, whereby the solution 
space breaks into an exponential number of well-separated clusters (this 
separation can be energetic or entropic). For even higher densities a second 
transition occurs, which reduces these clusters to a finite number. Finally, 
all clusters disappear at the SAT-UNSAT threshold. 

Since the clustering phenomenon is one of the main building blocks 
underlying the statistical physics approaclQ, substantial efforts have been 
made to give a rigorous base to it [TSjfTB] . Some mathematical results were 
also obtained in the ergodic phase [17j . and in the simple case of linear 
Boolean equations [18j . Remarkably, pure states play an central role in 
spin-glass theory, and they have been extensively studied in that context. 
However, the geometrical organization of glassy phases is not yet fully 
understood, and the classical picture of complex energy landscapes with 
many "valleys" still lacks an appropriate representation. 

In this paper we introduce an exactly solvable random-subcube modeH 
(RSM), in the spirit of Derrida's Random Energy Model (REM) '19\. This 
model is inspired by the structure of hard CSPs and optimization prob- 
lems, and reproduces most of their phenomenology. It can be also thought 
of as an attempt to construct a minimal setting that is able to reproduce 
the structure of solutions in hard CSPs. Its purpose is mainly pedagogi- 



^ Technically, clustering is closely related to th e o ne-step replica-symmetry 
breaking ansatz used in the replica/cavity method [14| |6]. 

^ Independently of our work A. Montanari inspired by an idea of D. Achlioptas 
also introduced this model and worked out some parts of our Sec. [2l 
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cal, and it offers an excellent testing playground for ideas and methods in 
combinatorial optimization and glass physics, while being fully tractable. 

Despite its simplicity, the RSM undergoes the same phase transitions as 
those observed in random CSPs such as /fc-SAT [6ll8] or fc-COL [8,10J. The 
connection even becomes quantitative in the large-fc limit of these problems. 
So far only the zeroth order of this limit was intuitively known and related 
to Shannon's random code model pUll^l^ , in which clusters are uniformly 
distributed singletons. One of our most notable results is that the RSM 
provides the first-order approximation of this large-fc limit, reproducing 
the cluster size distribution and freezing properties of the original models. 

We also generalize the RSM to deal with continuous energy landscapes 
resembling those observed in glassy systems and hard optimization prob- 
lems, and show how static and dynamical properties can be related explic- 
itly. This energetic RSM displays temperature chaos, undergoes a dynam- 
ical transition, and has a Kauzmann temperature. 

The paper is organized as follows: In Sec. O we define the model and 
describe its basic properties, as well as the connection with the random fc- 
SAT and fc-COL problems. We also analyze the behaviour of physics-guided 
decimation schemes. In Sec.[3]we discuss the relation between its dynamical 
and geometrical properties. In Sec. 3] we extend the definition to energetic 
landscapes, compute static and dynamical properties, and comment on 
some ideas from the physics of glassy systems. Finally we present a general 
discussion of our results in Sec. [5l 



2 The random-subcube model 

2.1 Definition 

Most constraint satisfaction problems are defined by a set of constraints 
on N variables cr = (tJi, . . . , ctat) with a finite alphabet, e.g. {0,1}. In 
contrast, the random-subcube model is defined directly by its solution space 
S C {0, 1}''^; we define S as the union of [2^^""^^] random clusters (where 
lx\ denotes the integer value of x). A random cluster A being defined as: 

A- V I V*e{i,...,7V},a, e^,^}, (1) 

where tt"* is a random mapping: 

^^:{l,...,iV}^{{0},{l},{0,l}} (2) 

^ ^ 7:f (3) 

such that for each variable i, nf ~ {0} with probability p/2, {1} with 
probability p/ 2, and {0, 1} with probability 1—p. A cluster is thus a random 
subcube of {0,1}^. If nf = {0} or {1}, variable i is said "frozen" in A; 
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otherwise it is said "free" in A. One given configuration cr migfit belong to 
zero, one or several clusters. A solution belongs to at least one cluster. 

The parameter a is analogous to the density of constraints in CSPs; 
clearly the SAT-UNSAT transition occurs at as := 1, where clusters cease 
to exist. The parameter p gives the probability that a variable is frozen, 
and plays a role similar to the clause size k in fc-SAT, or to the number 
of colors k in fc-coloring, as we will see later. Note that in the special case 
p = 1, the RSM is equivalent to the random code model with rate R = 1 — a 

EH. 

Frozen variables and the structure of the solution space have been in- 
troduced to mimic the situation observed in random CSPs. However there 
are important differences between the RSM and models like /c-SAT. First, 
in real CSPs the clusters are not necessarily subcubes of {0, 1}''^. We stress 
here that when speaking about clusters in the RSM we have in mind the 
above definition, whereas in the context of the CSPs the notion of cluster is 
more general [5 ,8 . Further, in real CSPs the sets of frozen variables associ- 
ated with clusters are correlated by the underlying graph, instead of being 
distributed uniformly. Moreover, free variables in CSPs do not enjoy the 
same freedom as in the RSM, as clusters usually do not fill up the whole 
subcube allowed by the frozen variables. In fact, free variables can be corre- 
lated within each cluster in a highly nontrivial way, and these correlations 
may even be so strong that they create clusters without the help of frozen 
variables. Clusters without frozen variables are indeed very important, as 
discussed recently in [Tnil23j . 



2.2 The basic structural phase transitions 

We now describe the static properties of the RSM in the thermodynamic 
limit N ^ oo (the two parameters a and p being fixed and independent 
of N). The internal entropy s of a cluster A is defined as log2 \A\, i.e. 
the fraction of free variables in A. The probability V{s) that a cluster has 
internal entropy s follows the binomial distribution 

ns) = ^^j^)ii-prp''-'^''. (4) 

Let 7V(s) be number of clusters of entropy s. This number follows a binomial 
law of parameter 'P(s) with 2^(^~") terms. Then the mean and the variance 
of A/'(s) read: 

m{s) = 2^(i-")p(s) , VarAA(s) = 2^(i-"¥(s)[l - P{s)] . (5) 
By Markov's inequality: 



P [Af{s) > 1] < E [Af{s)] , 



(6) 
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and by Chebyshev's inequality: 



, VarTVfs) 1 n „ „ 



(7) 

we get, with high probabihty (w.h.p.: with probabihty going to 1 as — > 
oo): 

lim llog,A^(.) = (^(^)-l-"-^(^l|l-^) ims)>0, 
N-^Qo N I — oo otherwise, ^ ' 

where D{x \\ y) :— x logj ^ + {l — x) logj is the binary KuUback-Leibler 
divergence. Throughout the paper, the same Markov/Chebyshev argument 
wiU apply every time we will have to deal with a number of clusters with 
a specific property. 

We now compute the total entropy Stot = jj log2 \S\. First note a ran- 
dom configuration belongs on average to 2^'^^~°\l — clusters. There- 
fore, if 

a < ad := log2 (2 -p), (9) 

then with high probability the total entropy is stot = 1- 

Now assume a > a^.- The total entropy is given by a saddle-point 
estimation: 

V2^(-^)'^ = [l + o(l)]A^ / ds 2^[^(^)+^l, (10) 



whence Stot = max [S{s) + s \ X!{s) > 0]. (11) 

We denote by s* = argmax[. . . ] the fraction of free variables in the clusters 
that dominate the sum. Note that in this sum solutions belonging to several 
clusters have been counted too many times. This does not affect the validity 
of our estimation, since in every cluster the fraction of solutions belonging 
to more than one cluster is exponentially small as long as a > a^. 

Define s := 2(1 — p)/(2 — p) such that dsS{s) = —1. The complexity of 
clusters with entropy s reads: 

U{S) = -^+log^{2-p)-a. (12) 
2 -J3 

s maximizes Eq. ([Tl]) as long as S{s) > 0, that is if 

a<ac:=^^^ + log2(2-p). (13) 
Then the total entropy reads 



Stot = 1 - a -I- log2 (2 - p) for a < ac- 



(14) 
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Fig. 1 (Color online) Top: graphical construction of the maximum of X'(s) -I- s 
by a Legendre transformation. In the top left figure (a < etc), the line of slope 

— 1 tangent to the complexity function gives the saddle-point s*, as well as the 
total entropy Stot by the intercept on the s axis. In the top right figure [a > Qc), 
the supporting line of slope —1 gets "stuck" at sm, where the derivative is — m > 

— 1. Bottom: represented as a function of q: total entropy Stot, total complexity 
Stot = 1 — a, typical entropy s*, complexity of dominating clusters 17* = 

and m = —dsS{s*). The condensation point Oc marks the separation between 
the two regimes illustrated above. 



For a > ac, the maximum in is realized by the largest possible cluster 
entropy sm, which is given by the largest root of S{s). Then Stot = s* = 
Sm- In this phase the dominating cluster^ have size e^* '^^j where A — 
0(1) is asymptotically distributed according to a Poisson point process of 
rate e""*"^, i.e., for dA -C A the probability that there is at least one state 
of size between 6^"*+-^ and 6^''*+'^+^^ is e-'"'^dZ\, where m = -ds^is*). 
Extreme value study of this process leads to the Poisson-Dirichlet 12411251126] 
distribution of weights of clusters. In particular it follows that an arbitrary 
large fraction of the solutions can be covered by a finite number of clusters. 
Such a phase is called condensed. 

In summary, for a fixed value of the parameter p, and for increasing 
values of a, four different phases can be distinguished: 

(a) Liquid phase, a < a^: almost all configurations are solutions. 

^ The "dominating clusters" are the minimal set of clusters covering almost all 
solutions. 
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(b) Clustered phase with many states, ad < a < etc' an exponential number 
of clusters is needed to cover almost all solutions. 

(c) Condensed clustered phase, ttc < a < = 1: a finite number of the 
biggest clusters cover almost all solutions. 

(d) Unsatisfiable phase, a > no cluster, hence no solution, exists. 

The very same series of phase transitions is observed in random fc-coloring 
and random fc-satisfiability, where a is the density of constraints [SlllOj. 
The condensation transition at ac corresponds to the Kauzmann temper- 
ature J^T] in the theory of glasses; at ac the total entropy Stot(a) has a 
discontinuity in its second derivative with respect to a, in analogy with the 
discontinuity of the specific heat at the Kauzmann temperature. 

Keeping in mind the similarity between the RSM and real CSPs, it can 
be useful to mention some of the properties that are commonly discussed 
in the statistical physics analysis of these problems (for brevity we omit the 
proves of these statements) . Among them is the probability distribution of 
mutual overlaps P{q), where q{a, a') = X]i=i[2<5(CTi, cr-) — 1]/N. Below the 
condensation transition, a < a^ we have P{q) = S{q), reflecting the fact 
that random pairs of solutions are uncorrelated. For a > ac, the overlap 
function consists of intra-cluster and inter-cluster overlaps: P{q) = w5[q — 
(1 — stot)] + (1 ^ 'w)S{q), where w is the sum of squares of weights of all 
the clusters, it is a non-self- aver aging random variable, the distribution of 
which can be computed from the Poisson-Dirichlet process [2511^ . 

An equivalent way of characterizing the condensed phase is to consider 
the fc-point correlation function, with k > 2: 

|P((7i = Xi,(T2 = X2, . . . ,(Tfe = a;fe) - P((Ti = a:i) . . . P(crA; = Xk)\. 

Xi,...,Xk 

(15) 

This quantity decays to zero as N goes to infinity in the non-condensed 
phase, whereas it remains bounded away from zero in the condensed phase. 



2.3 The large fc-limit of random /e-SAT and fc-COL 

One of the most interesting properties of the random-energy model [191 is 
its equivalence with the large-p limit of the p-spin glass [28 . In the same 
spirit, although the justification is slightly different, the random-subcube 
model is found to be equivalent to random fc-SAT, random fc-COL, and 
presumably other constraint satisfaction problems in the limit fc — > oo, 
for connectivities close to the satisfiability threshold. Let us detail this 
statement. 

It was already known that at zeroth order (when k —^ oo) random fc- 
SAT and fc-COL behave as a random-code model (random-subcube model 
with p = 1), in which clusters are uniformly distributed singletons. Recent 
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large-fc calculations of the cluster size distribution S{s) in fc-SAT and k- 
COL [SlITO] allow a direct comparison with the RSM. 
The control parameters of the RSM are rescaled as: 

p=l-s, " = l + £^' (16) 

with e ^ 1 and 7 — 0(1). The cluster size distribution in the RSM is then, 
at leading order: 

U{s) ln(2) = s [1 - In -1 - e(2 + 7) + o(e). (17) 

while the condensation and satisfiability thresholds read, in terms of the 
rescaled variable 7: 

7, = -21n2, 7s = -1. (18) 
In /c-SAT and /c-COL, identifying e and 7 in the following way: 

SAT: e=4-r, ^ ^ 2^ In 2 - + 2^ , (19) 
2fe+i 2 2 ^ ' 

COL: £=1, ^=fclnfc-i^ + 2, (20) 
2k' N 2 2' ^ 

where M is the number of constraints (edges in coloring) , and TV the number 
of variables, gives a perfect matcl|3 for the complexity function pT]) . as 
computed in [81ITU] . This illustrates the analogy between a and the density 
of constraints M/N, as well as between p and k. 

This equivalence goes further than simply having identical cluster size 
distributions. The cavity analysis of fc-SAT shows that the fraction of free 
variables in a cluster scales exactly as its internal entropy s. The entropy 
of clusters is thus maximal, from which we infer that clusters fill up the 
whole subcube prescribed by their frozen variables, like in the RSM. The 
same is true for fc-COL (compare Eqs. (E14) and (E27) in [10]) with the 
small difference that for every unfrozen variable only two (out of k) colors 
are allowed. 

Note that this comparison is valid only in a finite vicinity of the satis- 
fiability threshold, for 7 — 6*^(1). In particular, it does not encompass the 
clustering transition, which for fc-SAT (resp. fc-COL) occurs for constraint 
densities scaling as 2''ln2/fc (resp. fclnfc/2). 



* Note the difference in the logarithmic base between here and [8IIIO]. 
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2.4 Decimation 

An important contribution of statistical physics to the field of combinatorial 
optimization has been to exploit the information provided by message- 
passing algorithms to devise physics-guided decimation schemes. 

Message-passing algorithms exchange information between units (vari- 
ables and constraints) in order to obtain estimates of marginal probabilities 
(beliefs) or other related quantities (e.g. surveys, see below). Subsequently, 
this information is used to find a solution. A usual way to do this, called 
decimation, proceeds as follows: fix randomly the value of on^ variable ac- 
cording to its estimated belief, then re-run the message-passing algorithm 
on the reduced system, and loop. A trivial statement is that a perfect 
estimate of all marginal probabilities would always cause the decimation 
procedure to find a solution (if any). 

In the RSM there is no underlying graph, therefore message-passing can- 
not be defined. However, it is possible to study decimation schemes based 
on exact marginal probability estimators. Although such procedures should 
really be viewed as thought experiments, they can be used to gain some 
insight on real algorithms. Here two idealized algorithms are considered: 

— Belief estimator: outputs the exact marginal probabilities ^i{(Ji) = 
Ea,, m('t), where = l{a G S)/\S\. 

— Survey estimator: outputs "surveys" , i.e. marginal probabilities over the 
clusters: v,{t:,) = J2tv^^ f^(^). where i/(7r) = J2aH^ = tt^)/ [2^^'""^] ■ 

In real CSPs, belief and survey propagation arguably provide asymp- 
totically accurate estimators, as long as the number of clusters dominating 
the measure /x (or v for survey propagation) scales exponentially with N, 
and as long as the one-step replica symmetry breaking description is the 
correct one |141I10] . However, belief-guided and survey-guided decimation 
schemes are difficult to analyze in real CSPs — see [29] for an empirical 
study on surveys and [30] for recent analytical study on beliefs in /c-SAT. 

Let us study decimation in the RSM. As long as the phase is non- 
condensed, the belief estimator always outputs /ii(o'i) « 1/2 for all i in 
the limit N — > oo. Likewise, the survey estimator will output ^"^({0}) ~ 
j/idl}) Kip/2, i/i({0, 1}) w 1 — p. In both cases, the decimation procedure 
is completely unbiased: it will fix a random variable i to or 1, with proba- 
bility 1/2. This observation remains true in the subsequent decimation steps 
as long as the number of clusters dominating the reduced measure fj, (or 
ly) remains exponential. Within this assumption, after T = tN {0 < t < 1) 
decimation steps, T variables will be fixed randomly and independently. 
We are then left with a restricted space of solutions compatible with these 
T fixed variables. The logarithm of the number of clusters of entropy sN 

^ In practice the number of variables fixed at each step can range from one to 
a small fraction of the variables. 
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is then 



NUt{s)=N 



1-a + tlog, 



(21) 



Rescaling by the number of unfixed variables {1 — t)N: St = ^ t)^t, 
s = {I — t)s, we obtain 

St{-s)^l-^^-Dirs\\l-p). (22) 

The parameter a{t) := {a — tad)/(l — t) now plays the same role as a 
in the analysis of the RSM. Consequently, the system undergoes the same 
condensation and unsatisfiability transitions as t is increased. Assume for 
example that ad < a < olc- Fixing a fraction tc :— (etc — ce)/iac — ad) 
of the variables will cause the system to condense. The belief estimator 
will then be dominated by a finite number of clusters, yielding instance- 
dependent biases on variables. An extensive number of variables become 
suddenly near-frozen, i.e., /ii(l) ~ or « 1, and remain so for t > tc- At 
ts '■= (1 — Q;)/(1 — ad), the total complexity goes to zero, and near-frozen 
variables become truly frozen, i.e. /ii(l) = or 1, as all sub-dominant 
clusters disappear. By contrast, the survey estimator will output unbiased 
marginal probabilities as long as the total number of clusters is exponential 
in N, that is ii t < t^. At t = tg, decimation concentrates on a single 
cluster, causing a freezing avalanche in the surveys, i.e. for each variable 
i, fi({0, 1}) = 1 or i^i({0}) = 1 or i^i({l}) = 1. For both estimators, any 
slight error at tg will cause the failure of the decimation process. 

Note that in real CSPs, belief propagation is not expected to be correct 
in the condensed phase, beyond t^, in fact, it will not detect condensation, 
nor near-frozen variables. For a recent study of the belief-propagation- 
guided decimation in fc-SAT see [30]. Remarkably, in real CSPs, survey- 
guided decimation usually simplifies the problem: after a certain number 
of decimation steps, it outputs £^^({0, 1}) — 1 for all i, and the problem can 
then be passed over to a simple local search algorithm ^6,29]. Although the 
RSM, which is intrinsically clustered, is unable to capture this property, it 
sheds some light on why decimation may not work in some cases. 



3 Distance and ergodicity 

The geometrical organization of solutions to CSPs is thought to play an 
important role in setting intrinsic limits to the performance of search al- 
gorithms. In particular, the clustering phenomenon, by which the solution 
space is fragmented into many connected components far from each other, 
has been proposed by physicists as a possible explanation for the failure of 
most known algorithms [UlSllH]. Also, the role of frozen variables was re- 
cently discussed in [10] . Conversely, the success of survey propagation [6j is 
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usually explained by the fact that it explicitly incorporates the existence of 
clusters. The separability of clusters has been proved in the fc-SAT problem, 
in compliance with the predictions of statistical physics [T5| . Despite this 
evidence, the precise relation between geometry and algorithms still lacks a 
rigorous base. The random-subcube model offers an excellent opportunity 
to study these questions in a well controlled framework. 



3.1 The dynamical transition 

Let us first argue why the clustering transition at a^, defined by Eq. ([9]), 
actually corresponds to what is commonly refered to as the dynamical 
(ergodicity breaking) transition in real CSPs [31^.32,33.8, 12, 10 . To this 
end we study a uniform unbiased random walk on the space of solutions: 
at each step, one is allowed to move from one solution to the other by 
flipping only one spin. We choose this dynamics for the sake of simplicity, 
but most of the arguments below hold for more general dynamical rules, 
like for example the flipping of a sub-extensive number of spins at each 
step. 

We have already pointed out that an arbitrary configuration belongs 
to 2(i-")^(l different clusters w.h.p. if a < ad, and to none if 

a > ad- Therefore below ad almost all configurations are solutions, and 
any reasonable dynamics will explore the entire phase space uniformly. 

On the other hand, when a > ad, solutions become exponentially rare. 
Let A he a. cluster of internal entropy s. What is the probability that after t 
steps, a random walker ends up in another cluster B of internal entropy s'7 
Let a denote the proportion of variables that are free in A and frozen in B. 
The probability distribution of a is given by the total number of partitions 
of {1, . . . , N} into four categories: frozen in A and B, only frozen in A, only 
frozen in B, and frozen neither in A nor B. This probability reads: 

1 iV! 

~ (ns) (ns') iNay.[N{s ~ a)]![iV(l - ,s' - a)]![iV(.' - s + a)]\ ' 

In order for the walker to reach B from A, it has to match perfectly the 
prescriptions (freezings) of B on these aN variables. If t = 0{N'^), the prob- 
ability of this happening is « t2~°-^ . Additionally, variables that are frozen 
in both clusters must coincide, so that A and B have a non-empty intersec- 
tion. For a random choice of B, this happens with probability 2^^^* 

Consequently, the probability that the walker wanders in any other 
cluster of entropy s' after t steps is union-bounded by: 

(/)(s ^ s') < 2^^(^') * 2^""^ 2^(^'+'^-i) (24) 
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The maximum of jj log q{a) tends to zero when N ^ oo, so that: 

lim sup ^ log <j>{s ^ s') < S{s') + s' - 1 (25) 

This quantity remains negative for all s', as long as a > a^. Therefore, 
in this regime, hopping from one cluster to the other is very unlikely in 
a sub-exponential number of steps, even though clusters are not disjoint. 
In this sense we say that ergodicity is broken. Remarkably, the space of 
solutions becomes non-ergodic as soon as it becomes non-trivial, at ad- 

Let us stress the importance of the entropic barriers between clusters 
in our analysis. In the real CSPs and optimization problems the energetic 
barriers are usually thought of as more important, and clusters are even 
sometimes described as separated (by an extensive distance). The impor- 
tance of entropic barriers and the possibility of non-extensively separated 
clusters should, however, not be neglected in the studies of richer models. 



3.2 x-satisfiability 

The notion of a;-satisfiability was first introduced as a tool to study the 
geometrical structure of the solution space of CSPs [15]. An instance of CSP 
is said x-satisfiable if and only if it admits a pair of solutions separated by a 
Hamming distance ~ xN. In other words, x-satisfiability gives the distance 
spectrum of the solution space. This spectrum is estimated using three 
quantities: 

a) di = xi{a)N: the maximum distance between two solutions inside one 
cluster, 

b) 6,2 = X2{oi)N: the minimmn distance between two solutions from two 
distinct clusters, 

c) ds = x^{oi)N: the maximum distance between any two solutions (pre- 
sumably from two different clusters). 

The first of these quantities is estimated by noting that the maximum 
distance between any two solutions in a given cluster, i.e. its diameter, 
equals its entropy s. Therefore the maximum diameter/entropy xi is given 
w.h.p. by the largest internal entropy sa/, i.e. the largest root of S{s) — 
1 -a -£1(5 II l-p). 

Now take two clusters A and B at random, and consider the probability 
that their distance be xN . This distance is given by the number of variables 
which arc frozen in both clusters, but in a contradictory way, such that 
T^Aii) 7^ T^sii)- This happens independently with probability p^/ 2 for each 
variable, so that the number of such variables follows a binomial law of 
parameter p^/2. Therefore, the number A/'(x) of pairs of clusters at distance 
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Fig. 2 (Color online) The x-satisfiability threshold is constructed from the three 
distances x\{a), X2(a) and xz(pL). Below a threshold ctgap, distance spectra fail to 
detect the fragmentation of the solution space (there is no more "gap" between 
intra and inter-cluster distances). Below another threshold Osop, clusters cease 
to be all well separated, although ergodicity is still broken. The condensation 
threshold a.^ is shown for information. The dynamical threshold lies outside 
the picture, and its value is ~ 0.070. In this figure p = 0.95. 



xN coincides w.h.p. with its mean valtie: 

E[AA(.)]=22(i-")^(^^J(^l-|-j (^-^ x2^-(-) (26) 

if S2{x) := 2(1 -a)- D{x \\ /2) > 0, and 7V(x) = w.h.p. if 52(2;) < 0. 
Consequently the smallest possible distance between any two clusters is 
given by X2N, where X2{a) is the smallest root of S2{x). A similar argument 
gives the largest distance between any two solutions from two different 
clusters: X3{a) = 1 — X2{a). 

To sum up, we find that a random instance is x-satisfiable w.h.p. if 
a < as{x), and is a;-unsatisfiable w.h.p. if a > as{x), with: 

if a; e [0, 1 - p] U 1 - 

D{x\\l-p) iixe[l-p,xo] , , 

kD(x\\py2) ifxG [xo,pV2] ^ ' 

Id{1-x\\p^/2) ifxe [1-pV2,1] 

where xq is solution to D{x \\ p'^/2) = 2D{x \\ l~p). 



as{x) = 
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Fig. [2] shows how as{x) can be constructed from the three distances xi, 
X2, X3. We put asop 1 + (l/2)log(l — > ad, the threshold below 

which some pairs of clusters have a non-empty intersection. An interesting 
observation is that ergodicity can still be broken even below this threshold. 
We can also define the agap := as{xo) > Uscp, below which distances 
from the same cluster and distances from distinct clusters overlap. This 
threshold sets the limit below which the notion of ^-satisfiability fails to 
detect clustering. The random-subcube model allows us to make a clear 
and intelligible distinction between the three thresholds ad < asep < Qfgap. 
We expect this distinction to hold in most CSPfH. 



4 Random energy landscape 

4.1 Definition 

The random-subcube model can be enriched by adding the notion of energy 
to the definition of states. The motivation for doing this is to mimic the 
optimization version of CSPs (where energy is defined as the number of un- 
satisfied constraints), but it can also be used to reproduce some properties 
of glassy systems. 

For each energy level Eo, we define J\f{Eo) = [2^^(-'=^«/^)J valleys of 
energy Eq, where S{eo) is an increasing complexity function. This function 
can be arbitrary, but for simplicity we will restrict our examples to the 
forn{l| S{eo) = a + beo — ceoln(eo), where a > corresponds to a SAT 
phase, and a < to an UNSAT phase. 

Each valley is defined as a subcube V, chosen at random in the same 
way as clusters in the previous sections, cf. ([1]). In the following, the freezing 
probability p will be fixed for all energies, but one could easily generalize 
the model by making it energy-dependent: p = p{eo)- 

The number M{Eo,So) of valleys of energy Eq = bqN and entropy 
Sq = sqN is w.h.p.: 

J^{Eo,So) - 2^^(^«>^«) if U{eo, so) := ^(eo) - D{so 1| f - p) > 
= otherwise. 

Given a configuration <t, we define its energy as a trade-off between the 
energy of surrounding valleys and their distance. Let us denote the energy 
of a valley V by Eo{V). Then the energy of cr is: 

E{(t) -.^imiilEoiV) +d{(T,V)] (29) 

® With the notable exception of fc-XORSAT, where Od = Qfsop. Incidentally in 
fc-XORS AT we also have Uc = as. 

This form corresponds to the function used to fit data from the cavity method 
in the fc-SAT problem [gj. 
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where c?(cr, V) is the distance between cr and the nearest element of V. 
By definition, we say that cr belongs to the basin of attraction of V if 
V minimizes the sum. Observe that with this definition, it may happen 
that some valleys are not represented at all in the energy landscape. In the 
following, the term "state-bottom energy" shall refer to the energy Eq of 
the valley minimizing the sum, and the term "state-bottom entropy" to the 
entropy of that valley. 



4.2 Static description of the energy landscape 

What is the energy of an arbitrary configuration cr? Let us start with 
the typical case: for each eo = Eq/N, we compute the distance to the 
nearest valley with energy Eq. Standard arguments show that the number 
of valleys of state-bottom energy eoA'' at distance d = uN is governed by 
the exponent: 

S{eo) - D{u II p/2). 
Then, the minimum distance is given, in the N oo limit, by 

S[S{eo),p/2]N, where S{x,y) is solution to: x = D(5 \\ y). Then, the typi- 
cal energy is obtained as the best compromise between state-bottom energy 
and distance: 

e* = E/N = min{eo + 6[E{eo),p/2]} , (30) 

eo 

The argmin gives the typical state-bottom energy of a random cr. 

As we just saw, most configurations have roughly the same energy, and 
belong to valleys with the same state-bottom energy. At finite temperature 
however, thermodynamics will be dominated by configurations of lower 
energy than e* A''. We thus need to estimate the entropy function (governing 
the number of configurations of given energy) , the Legendre transform of 
which shall give us the free energy. Given a valley V of energy Eq = cqN 
and entropy So = sqN, the number of configurations of energy E belonging 
to this valley is: 

^ 2^-v(e|eo,.o) ^ith sv{e\eo,so):=so + {l-so)H(^^ 

V 1 - •So 

where H{x) = —x logj x — (1 — x) log2 (1 — .x) is the entropy function. Sum- 
ming up over all valleys, the total number of configurations with energy 
E = eN is: 



N-So 



with s(e) = max [sy (ejeo, so) + -^(eo, so)] • (33) 

i;(eo,so)>0 
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Here we have implicitly assumed that all elements in the sphere of radius 
E — Eq and center V are in the basin of attraction of V, as long as E < e*N. 
This is not true in general, as some configurations may in fact belong to 
a more favorable basin, and may thus have lower energies. However, such 
configurations remain exponentially rare in comparison to the total weight 
of the sphere. Therefore the previous estimate holdfH. 

Canceling the derivative w.r.t. sq in ([55]) yields the saddle for sq: 

, (l-p)(l-e + eo) 

= W2 ^^^^ 

Provided that the maximum is reached in a region where S{eQ^ sq) > 0, we 
get: 

s(e) = max [1 - D(e - eo || p/2) + E{eo)] for < e < e*. (35) 

This will be valid from e — e* (for which we find s(e*) = 1 as expected) 
down to a certain condensation energy Cc. Below that energy (e < Cc), the 
phase is condensed: E{eo,so) < 0, and the maximum in (|33p is reached on 
the border on the definition domain, where i7(eo, so) = 0. If we denote by 
Sj\/(eo) the biggest valley of energy eo (i.e. the largest of root of S{eo, so) = 
0), we get: 

s(e) = max{sy [e|eo, SAf (eo)]} for e < Cc- (36) 

So far we have worked in the microcanonical ensemble, but the same ar- 
guments hold in the canonical ensemble. In particular the condensation 
temperature is Tc = {ds/de\e=ea) ^- The number of dominating states in 
the condensed phase follows again a Poisson-Dirichlet process [251126] with 
parameter m, where m/T is the slope of the curve Sxif) at its smallest 
root. The function Srif) is the canonical counterpart of Z'(eo,so): and 
reads: 

EtU) = max 17(60,30), (37) 

eo,so:/v(T|eo,so)=/ 

where /y(T|eo, so) is the single-state free energy, obtained as the Legendre 
transform of sy(e|eo, sq) in ([31]) . 

In both low-temperature phases (condensed and non-condensed), equi- 
librium is reached for different values of (eo, so) as e varies. Said differently, 
the states dominating the microcanonical measure at e and at e -\- Se are 
completely distinct. In the canonical language, we say that the system ex- 
hibits temperature chaos [341l35j : slightly changing the temperature from T 
to T + 5T dramatically modifies the free-energy landscape, reshuffling the 
ordering of states. Consequently, correlations are nonexistent between T 

^ This argument is similar to the one used in Eq. (|10p . where some solutions 
were counted several times, but with no consequence at the exponential scale. 
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Fig. 3 (Color online) Illustration of the temperature cha os b y construction of the 
microcanonical entropy in the condensed phase, cf. Eq. (|36|) . and level crossing. 
We have represented three entropy curves sv{e) corresponding to three extremal 
states Vi , V2 and V3 , the envelope of all these curves is the microcanonical entropy. 
These states are maximally atypical; they realize a balance between low state- 
bottom energies and a high state-bottom entropies, which are related by sq = 
SM(eo) (thick curve). As the temperature (or energy) is decreased, the curves 
sv{e) cross each other, and the system is dominated by states of lower state- 
bottom energies and entropies. These data were obtained for number of valleys 
Sieo) = -0.05 - 0.5eo In eo and p = 0.2. 



and T + ST. This phenomenon of free-energy crossings is iUustrated (in the 
condensed phase) by Fig. [31 where the maximum of Eq. ([55)1 is constructed 
geometrically. 

4.3 Relation with dynamics 

We now undertake to describe the dynamical properties of this energetic 
landscape. To that end we shall make use of the static picture, which we 
know precisely from bottom-up construction. 

Our reasoning proceeds in two steps. First, we study the behavior of 
a single spin-flip Monte-Carlo dynamics with detailed balance evolving in 
a single valley of state-bottom energy Cg, and state-bottom entropy Sq. In 
a second step, we argue that the same dynamics run on the full rugged 
energy landscape is entirely governed by this single-state-like behavior. 

The thermodynamics of a single typical state is given by: 

sie\el,s*) = .s* + il-s*)H(^^-^^ (38) 
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where Sq = (1— — e* +e|5)/(l — p/2) is the typical state-bottom entropy 
of a random configuration, computed from Eq. ([34|) . Supposing now that the 
energy landscape is made of this state only, we can easily convince ourselves 
that the dynamics is ergodic: the energy landscape is convex, and there are 
neither energetic nor entropic barriers. Both the quenched and annealec^l 
dynamics in this simple landscape is then given by the thermodynamics. 

We know from the previous paragraphs that states extend up to energy 
e*, where a "crest" connects the different valleys. This crest is actually more 
of a plateau, as it embeds almost all configurations: s(e*) — 1. Therefore, 
our dynamics will remain there as long as the temperature does not al- 
low configurations of lower energy. This happens at := {ds/de\e=e') ^, 
where exploring valleys starts to be more favorable. 

Below Td, the system will find itself trapped in one state, since barri- 
ers between valleys are extensive. This randomly picked state has typical 
properties: in particular, its bottom-energy is Cq and its bottom-entropy is 

Sq. 

We argue that below Td everything happens as if this trapping state was 
put in isolation, as we have described above. The justification comes from 
the fact that a valley does not "see" its neighbors as long as e < e*: even 
though many configurations of energy e < e* in the isolated state actually 
belong other valleys in the full landscape, their proportion is exponentially 
small. In other words, although there may be some directions for which the 
energy barrier is lower than e* — e, these directions are beaten entropically. 

With this reasoning, both the quenched and annealed dynamics are 
described by the microcanonical entropy of a single typical state: Sdyn(e) = 
s(e|e5,So). We note in passing: 

r-i = — 
de 

as partial derivatives w.r.t. eg and s inside (|33p cancel at the maximum. 
The dynamical temperature Td is thus well defined. 

Note that this analysis is exact: the fact that it purely relies on static 
arguments should be attributed to the simplicity of the model. In richer 
mean-field models this kind of arguments might also apply, but will most 
probably not give the full picture. 



dsd 



yn 



de 



(39) 



4.4 Glassy behaviour 

The analysis of the energy landscape in the RSM is summarized in the 
energy-temperature phase diagram of Fig.|4l As anticipated, the behaviour 

® By quench, resp. annealing, we mean a fast, resp. slow, change of temperature 
from 7i > Td to T2 < Td and some time (a finite number of Monte Carlo sweeps) 
spent at T2 afterwards. 
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Fig. 4 (Color online) Energy as a function of temperature. At temperature 
T > Td, the system is in a liquid state: the dynamics is exploring ergodically 
all configurations of energy e*. Below the dynamical temperature ergodicity is 
broken. The upper "dynamical" curve shows the result of a quench/annealing 
in temperature, whereby the systems remains stuck in a typical state of bot- 
tom energy Bq. For Tc < T < Td, equilibrium thermodynamics is dominated 
by an exponential number of states (curve "glass"). Below the condensation 
(Kauzmann) temperature thermodynamics is dominated by a finite number of 
states (curve "condensed glass"). This number is given by a Poisson-Dirichlet 
process of parameter m (plotted in the upper part of the diagram). The dashed 
line shows the result of a quench/annealing starting from an equilibrium state 
at temperature Tc < T < Td- The bottom line shows the bottom energies 
of the thermodynamically dominating states. These curves were obtained for 
S{eo) = -0.05 - 0.5eo In eo and p = 0.6. 



of the RSM resembles the one observed in glasses and spin-glasses. The 
two distinct glassy transitions (dynamical and condensation), as well as 
the phenomenon whereby the physical dynamics gets stuck in metastable 
states, have been described for example in thep-spin glass [35], the spherical 
p-spin glass [S^, the Potts glass [37j and the lattice glass [38]. Several 
related examples of energy-temperature diagrams were derived recently in 

m- 

In the aforementioned mean-field models, the static behaviour is bet- 
ter understood than the dynamics. Static properties are usually analyzed 
by the replica/cavity method, with the help of Parisi's replica symmetry 
breaking scheme. A satisfactory analytic treatment of the dynamics exists 
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only for the spherical p-spin glass [36], in which all states have the same 
entropy. A remarkable step towards connecting the static picture and dy- 
namical behaviour in a rather general framework was done in |311I32[[33] . 
The RSM could provide a tractable playground for studying several aspects 
of glassy dynamics, e.g. aging and rejuvenation [40j . 

We would like to emphasize the freedom we can enjoy in the definition 
of the energy landscape. First, arbitrary numbers and sizes of valleys at 
each energy eo, ^(eo) and p(eo), can be considered. Second, the definition 
of the configurational energy E{(t) in Eq. (I29|) could be generalized to an 
arbitrary function of all valleys V and cr; E{cr) = !F{{V}, a). By tun- 
ing these parameters, one could hope to reproduce the dynamics of more 
complex models on a quantitative level. 

5 Conclusions 

The random-subcube model is a simple exactly solvable model capturing 
several interesting properties of random constraint satisfaction problems. 
Rather than an attempt to construct a new realistic model for practical 
instances of constraint satisfaction problems, it allows us to identify which 
properties of random CSPs can be reproduced by a simple probabilistic 
structure, and conversely, which of these properties may be intrinsically 
non-trivial. Examples of reproducible properties include condensation, non- 
monotony of the a;-satisfiability threshold, temperature chaos and dynami- 
cal freezing in metastable states. From this point of view, the RSM stands 
just next to the random-energy model [19], the random-code model [20[|2T[ 
122] or the random-energy random-entropy model [35j . 

Since the relation between the RSM and the large-A: limit of random 
fc-SAT and fc-COL is based on non-rigorous results from [8,10j, it would 
be interesting to establish this equivalence rigorously. Further, the RSM 
should be helpful for understanding some properties which are too difficult 
to study in more realistic models, such as finite-size corrections or some 
aspects of glassy dynamics. 

Finally, our work addresses the broad question of producing, infer- 
ring, and representing complex and rugged structures of the hypercube. 
Although we retained the simplest choice of subcubes for clusters, more so- 
phisticated alternatives could be explored and used to reproduce detailed 
geometrical features of solution spaces in CSPs. 
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